Exploring the Gap between a Pre- and Post-installation of a Corporate E-learning Program in an Accounting Workplace

Although earlier concentration has addressed the use of corporate e-learning programs (CELP), the dissimilitude between pre and post installation reaction to CELP is less explored. This study adopted a two-phase investigation to survey learner intention to use CELP and actual behavior within an international accounting firm. In the pre-installation phase, a survey was conducted to evaluate learner intention to use CELP, followed one year later by a second investigation to examine learner reactions to CELP and the actual usage frequency/ duration. The results of this study identified there is actually a difference between intention and actual usage duration. Further questionnaire surveys were implemented to identify learner reactions and factors that could potentially contribute to the gap between intention and actual usage duration. Results also indicated that scheduling was the critical factor leading to the differences in actual usage. The conclusions clarify the relationships among learner intention to use CELP, actual usage frequency/ duration, and subsequent reactions towards it. INTRODUCTION Among the many available forms of training media, e-learning has been widely adopted for obtaining skill-based organizational outcomes (Welsh, Wanberg, Brown, & Simmering, 2003; Zhang, 2004). Many financial institutions invest in e-learning programs to help their employees acquire knowledge regarding new services to better meet the diverse requirements of their customers (Luor, Hu, & Lu, 2009). A report published by Learning Circuits and E-learning News (2008) showed that corporate employees represent the majority (73.2%) of e-learning users. This figure highlights the attempts of businesses to develop their core competencies through low-cost, convenient, and flexible e-learning mechanisms. To aid in employee training, employers install and make e-learning programs available on the internet. Easy access to programs such as these can strengthen an institution’s competitiveness, especially if the market in which the institution operates is highly competitive. In Taiwan, the accounting industry is facing just such competitive pressure in terms of service quality and administrative efficiency (Wu, 2008; Cheng, 2011). Researchers have noted that Taiwan has a high-performing society that encourages employees to improve their performance and rewards them for excellence (Javidan & House, 2001; Luor, Lu, Johanson, & Wu, 2009). Like any training program, corporate e-learning programs (CELPs) encounter the difficulties of attracting, satisfying learners and keeping them engaged until the completion of the program (Noe, 2002; Wang & Wang, 2004; Lee, Tseng, Liu, & Liu, 2007). As with the success of any organizational policy, success in implementing CELP depends largely on employee participation (Wang & Wang, 2004). If learners perceive that courses are optional or have little positive effect on learners, lower completion rates are a likely result (Welsh et al., 2003). By analyzing learner reactions to CELP, executives can determine whether the learners accept the program; in other words, learner comments regarding the program can help to improve future implementation (Weibel, Stricker, & Wissmath, 2012). It is also important that CELP directors are aware of both favorable and unfavorable reactions to ensure that factors contributing to the success or failure of training activities are revealed (Stricker, Weibel, & Wissmath, 2011). To capture the entire spectrum of CELP, this study adopted both formative and summative approaches to examine learner reactions to CELP before and after installation of the program separately (Mohr, 1995; Luor et al., 2009) For the formative evaluation, a questionnaire was adopted to analyze the processes of the program (Mohr, 1995). For the summative evaluation, another questionnaire was developed to concern a program's effect on the development of knowledge and skills (Kirkpatrick, 1996; Luor et al., 2009). Previous research has provided evidence that learners in an e-learning environment outperform their counterparts in traditional learning settings (Shachar & Neumann, 2010; Stricker et al., 2011; Weibel et al., 2012). In addition, Luor et al. (2009) further proposed a gap between the learner intention and actual usage of CELP. However, few studies have explored how different factors influence the actual behavior to CELP within the contextual relation of a practical workplace at the same time. In short, this study had two goals. First, a framework was developed and tested based on the variables of intention to use CELP (INT), actual usage freqency (AUF), actual usage duration (AUD), perception of utility (UT), satisfaction (SAT) and affective reaction to e-learning (AR). This frame was used to provide a better understanding of how learner intention influences the actual usage of CELP and helped to identify the rationale behind the observed relationships. TOJET: The Turkish Online Journal of Educational Technology – October 2013, volume 12 issue 4 Copyright © The Turkish Online Journal of Educational Technology 81 Second, doubts have been raised as to whether a gap actually exists between INT, AUF, and AUD. Moreover, a further questionnaire survey was conducted to find out the critical factors that may contribute to this gap. METHOD Hypothesis INT and AUF/ AUD Based on the theory of reasoned action (TRA; Fishbein & Ajzen, 1975) and the theory of planned behavior (TPB; Ajzen, 1991), learner intention to participate in a specific behavior derives from their attitude to that behavior. A lot of studies have proposed the relationship between intention and behavior (Kim & Hunter, 1993; Luor et al., 2009). These theories suppose that actual behavior draws from intention to use (Fishbein & Ajzen, 1975, Ajzen, 1991, Taylor & Todd, 1995). Both TRA and TPB regard intention as the precedent of actual behavior. That is, only learners with intentions toward a particular behavior are likely to follow through. In the contextual relation of CELP, learner INT is positively related to actual usage (AU). AU in this study adopted two methods for calculation: (1) learner log-in frequency (AUF); and (2) learner review duration (AUD). According to TRA and TPB, the following hypothesis was proposed. Hypothesis 1: Learner INT is positively related to learner AUF, such that the higher the learner INT, the higher the learner AUF. Hypothesis 2: Learner INT is positively related to learner AUD, such that the higher the learner INT, the longer the learner AUD. AUF/ AUD , UT, SAT, and AR Four criteria are commonly thought as proper for training evaluation: reactions, learning, behavior, and results (Kirkpatrick, 1959 & 1996; Luor et al., 2009). Moreover, reaction is the most frequently measured consequence in practice among them (Brown, 2005). Reaction is defined as how learners perceive a CELP, including the learning content and structure, learning strategies, facilitated design, information and interface design, and learning assessment and feedback in this study. Evaluating a CELP in terms of learner reaction is the same as measuring trainee feelings (Kirkpatrick, 1996). From the viewpoints of CELP designers, reactions are effective because they authorize organizations to evaluate learner satisfaction towards program. This evaluation may in turn help the organization to determine the courses to offer and the instructors to appoint. In measuring learner reaction, this study adopted the three types of reactions proposed by Brown (2005) and Luor et al. (2009): UT, SAT and AR. UT refers to the degree to which learners feel the CELP is useful (Brown, 2005; Luor et al., 2009). Learners with high UT tend to use what they learned at work and suggest the CELP to their colleagues (Orvis, Fisher, & Wasserman, 2009). SAT directs to the degree to which learners are satisfied with the environment in which corporate e-learning courses are delivered (Brown, 2005; Liaw, Huang, & Chen, 2007). Even though this point is not directly related to the content of the CELP, it plays an important role because technical problems can reduce the likelihood that learners will participate in e-learning (Luor et al., 2009). AR refers to the general affective feeling that learners have toward the CELP (Brown, 2005; Luor et al., 2009). This type of reaction can often affect the retention of CELP. Thus, the following hypotheses were proposed. Hypothesis 3: Learner AUF is positively related to learner UT, such that the higher the learner AUF, the higher the learner UT. Hypothesis 4: Learner AUD is positively related to learner UT, such that the longer the learner AUD, the higher the learner UT. Hypothesis 5: Learner AUF is positively related to learner SAT, such that the higher the learner AUF, the higher the learner SAT. Hypothesis 6: Learner AUD is positively related to learner SAT, such that the longer the learner AUD, the higher the learner SAT. Hypothesis 7: Learner AUF is positively related to learner AR, such that the higher the learner AUF, the higher the learner AR. Hypothesis 8: Learner AUD is positively related to learner AR, such that the longer the learner AUD, the higher the learner AR. Participants The participants were employees at an international accounting firm in Taiwan. This firm is among the top four accounting firms in Taiwan. A total of 330 Level I employees from the auditing department participated in this study. The CELP comprised four e-courses, including pre-audit meeting, audit practice I, audit practice II, and post-audit review. The four e-courses were designed and developed with the cooperation of this firm and the researcher, using the ADDIE method. TOJET: The Turkish Online Journal of Educational Technology – October 2013, volume 12 issue 4 Copyright © The Turkish Online Journal of Educational Technology 82 Measures This study explored learner intention and actual usage to CELP before and after the installation of this program separately. To achieve this objective, this study measured participant intention to use CELP (INT), utility (UT), actual satisfaction (SAT), and affective reaction (AR) to CELP on the basis of items from the scale by Fishbein & Ajzen (1975) and Brown (2005). All items were measured on a 5-point Likert scale ranging from 1 to 5 (strongly disagree to strongly agree). The items used in this study and the reliability coefficients of each construct are shown in Table 1. All items were adopted from well-developed scales that have been used extensively in previous studies (Cacioppo & Petty, 1984; Fishbein & Ajzen, 1975). Because the reliability coefficients exceeded .70 and because the proposed relationships among the study variables were consistent with the relationships indicated by previous research (Schmidt & Hunter, 1996; Luor et al., 2009), the psychometric properties of the items in this study should be acceptable (Kim & Hunter, 1993; Schmidt & Hunter, 1996). To ensure the content validity of the scale (Schmidt & Hunter, 1996), ten individuals participated in the pre-test according to Cheng (2011). Five participants were employees with experience in e-leaning at the firm in the case study. Each item in the questionnaire was evaluated using a three-point Likert scale, including A as “Unclear question”, B as “Needed modification”, and C as “Clear question”. Usually, the items that had an A-point were deleted, the items that had a B-point were revised, and the items that had a C-point were kept for the questionnaire. The five other participants included three experts in accounting and two experts in e-learning. Each item in the questionnaire was also assessed using a three-point Likert scale, with 1 as “It is not necessary to ask the question”, 2 as “It is useful, but not essential to ask the question”, and 3 as “It is essential to ask the question”. The questions that received a one-point score were deleted, the questions that received a two-point score were revised, and the questions that received a three-point score were kept in this study. Table 1. Measurement and Reliability Measurement items Cronbach’s α 1.Intention towards corporate e-learning (INT) I was intent on using the corporate e-learning program.